[MOD-109][Feature] Reviews, Volume I: Moderation#7708
Conversation
9f21d66 to
f4787fb
Compare
| ## command: /bin/bash -c "cd /ember-osf && yarn link && cd /code && yarn link @centerforopenscience/ember-osf && yarn --pure-lockfile --ignore-engines && ./node_modules/bower/bin/bower install --allow-root --config.interactive=false && ./node_modules/ember-cli/bin/ember serve --host 0.0.0.0 --port 4300" | ||
|
|
||
| reviews: | ||
| #build: ../ember-osf-reviews # remove me before committing |
There was a problem hiding this comment.
I believe this block needs to be commented out. Also, assuming this comment is correct, this line should be removed.
| # sync_excludes: ['.DS_Store', '*.map', '*.pyc', '*.tmp', '.git', '.idea', 'bower_components', 'node_modules', 'tmp', 'dist'] | ||
| # watch_excludes: ['.*\.DS_Store', '.*\.map', '.*\.pyc', '.*\.tmp', '.*/\.git', '.*/\.idea', '.*/bower_components', '.*/node_modules', '.*/tmp', '.*/dist'] | ||
|
|
||
| reviews-sync: |
| # Full permissions: all routes intended to be exposed to third party API users | ||
| FULL_READ = NODE_ALL_READ + USERS_READ + ORGANIZER_READ + GUIDS_READ + METASCHEMAS_READ + DRAFT_READ + (CoreScopes.INSTITUTION_READ, CoreScopes.SEARCH, ) | ||
| FULL_READ = NODE_ALL_READ + USERS_READ + ORGANIZER_READ + GUIDS_READ + METASCHEMAS_READ + DRAFT_READ + REVIEWS_READ + (CoreScopes.INSTITUTION_READ, CoreScopes.SEARCH, ) | ||
| FULL_WRITE = FULL_READ + NODE_ALL_WRITE + USERS_WRITE + ORGANIZER_WRITE + DRAFT_WRITE |
There was a problem hiding this comment.
Should REVIEWS_WRITE be added here?
There was a problem hiding this comment.
That makes sense, yeah.
|
I'm getting an error when running |
|
Aha, this seems to fix it diff --git a/osf/migrations/0060_reviews.py b/osf/migrations/0060_reviews.py
index 8462b26682..b95512c321 100644
--- a/osf/migrations/0060_reviews.py
+++ b/osf/migrations/0060_reviews.py
@@ -3,6 +3,7 @@
from __future__ import unicode_literals
from django.conf import settings
+from django.core.management.sql import emit_post_migrate_signal
from django.db import migrations, models
import django.db.models.deletion
import osf.models.base
@@ -11,6 +12,8 @@ from reviews.permissions import GroupHelper
def create_provider_auth_groups(apps, schema_editor):
+ # this is to make sure that the permissions created in an earlier migration exist!
+ emit_post_migrate_signal(2, False, 'default')
PreprintProvider = apps.get_model('osf', 'PreprintProvider')
for provider in PreprintProvider.objects.all():
GroupHelper(provider).update_provider_auth_groups() |
|
@sloria Nice! Don't know why that didn't break in the same way for me, but I'll fix that up. |
18621c9 to
6ae51f6
Compare
brianjgeiger
left a comment
There was a problem hiding this comment.
Overall looks great from my perspective. A few things here and there, but no show-stoppers for me.
| super(ReviewableCountsRelationshipField, self).__init__(*args, **kwargs) | ||
|
|
||
| def get_meta_information(self, metadata, provider): | ||
| # Clone metadata because it's mutablity is questionable |
There was a problem hiding this comment.
Trivial: its not it's.
| return target.reviews_submit(user) | ||
| except InvalidTransitionError: | ||
| # Invalid transition from the current state | ||
| raise JSONAPIAttributeException(attribute='trigger', detail='Cannot trigger "{}" from state "{}"'.format(trigger, target.reviews_state)) |
There was a problem hiding this comment.
This should probably be a 409 conflict. At a future todo (and for the one below), it would be awesome if it could also tell the user what the valid transitions are from this point.
|
|
||
|
|
||
| class CreateAction(JSONAPIBaseView, generics.ListCreateAPIView): | ||
| """Create Actions *Write-only* |
There was a problem hiding this comment.
I think we've discussed, but more important than this is documentation at http://github.com/centerforopenscience/developer.osf.io
There was a problem hiding this comment.
I agree, I'll update my WIP PR there next week.
There was a problem hiding this comment.
Do we expect this API to be stable? If nah, it shouldn't go to developer.osf.io quite yet.
There was a problem hiding this comment.
Good point, though it probably shouldn't go here, either. @aaxelb you'll be getting an invite to an osf project which will be a better place to hold this info until it's stable.
|
|
||
| trigger = serializer.validated_data['trigger'] | ||
| permission = reviews_permissions.TRIGGER_PERMISSIONS[trigger] | ||
| if permission is not None and not self.request.user.has_perm(permission, target.provider): |
There was a problem hiding this comment.
Does reviews_permissions.ActionPermission not catch this?
There was a problem hiding this comment.
Since it depends on the trigger value I put this check after the serializer had validated. Do you think it would be better to have some validation logic in ActionPermission (and raise 400 if need be), or to have some permission logic in the view? I guess the former seems cleaner...
There was a problem hiding this comment.
If possible, it's better to have permissions checks in the permissions classes for detail and action views. If not possible, or really messy, then do what you need to do.
| raise PermissionDenied(detail='Performing trigger "{}" requires permission "{}" on the provider.'.format(trigger, permission)) | ||
|
|
||
| if not target.provider.is_moderated: | ||
| raise Conflict('{} is an unmoderated provider. If you are an admin, set up moderation by setting `reviews_workflow` at {}'.format( |
|
|
||
| published = validated_data.pop('is_published', None) | ||
| if published and preprint.provider.is_moderated: | ||
| raise Conflict('{} uses a moderation workflow, so preprints must be submitted for review instead of published directly. Submit a preprint by creating a `submit` Action at {}'.format( |
| def url(self, preprint_provider): | ||
| return '/{}preprint_providers/{}/'.format(API_BASE, preprint_provider._id) | ||
|
|
||
| def test_update_reviews_settings(self, app, preprint_provider, url, admin, moderator): |
There was a problem hiding this comment.
This was a good candidate for a single large test. I approve.
| res = self.app.post_json_api(self.url, private_project_payload, auth=self.user.auth) | ||
| assert not mock_on_preprint_updated.called | ||
|
|
||
| @mock.patch('website.preprints.tasks.get_and_set_preprint_identifiers.si') |
There was a problem hiding this comment.
Not specific to this PR, but my general comment with mocks is to double-check to make sure that it is actually testing a thing. Since you're getting a 409 below, it probably is, but these are tricky and have sat around doing nothing before, so just in case.
There was a problem hiding this comment.
I copied it from the other tests, looks like it's not actually testing a thing.
| assert expected == actual | ||
|
|
||
| # order by date_last_transitioned | ||
| # TODO once sorting is fixed |
There was a problem hiding this comment.
Obvs this should be set to go before you merge.
| preprint.save() | ||
| bad_payload = self.create_payload(preprint._id, trigger=trigger) | ||
| res = app.post_json_api(url, bad_payload, auth=moderator.auth, expect_errors=True) | ||
| assert res.status_code == 400 |
There was a problem hiding this comment.
When you fix the item from the top, this will turn into a 409. The request formatting is valid, it just conflicts with an existing database state.
| PreprintProviderFactory, | ||
| ) | ||
|
|
||
| from tests.base import ApiTestCase |
There was a problem hiding this comment.
Minor: Remove unused imports.
| from api.base import permissions as base_permissions | ||
|
|
||
|
|
||
| class ActionMixin: |
There was a problem hiding this comment.
This class seems unnecessary. ActionMixin.actions_queryset can be made into a function, something like get_actions_queryset().
There was a problem hiding this comment.
It was a function originally, but quoth @chrisseto:
I'd vote for a mixin here rather than having a functions just floating around.
But he quit so function it is!
| ).filter(is_deleted=False) | ||
|
|
||
|
|
||
| class ActionDetail(JSONAPIBaseView, generics.RetrieveAPIView, ActionMixin): |
There was a problem hiding this comment.
No need to inherit from ActionMixin.
| # -*- coding: utf-8 -*- | ||
| from __future__ import unicode_literals | ||
|
|
||
| from enum import Enum |
There was a problem hiding this comment.
Is enum34 explicitly declared as a requirement, or is it a sub-dependency? I can't seem to find where it comes from.
There was a problem hiding this comment.
Oh, I just used it and it worked. I'll add it as a requirement.
|
|
||
| if request.method in permissions.SAFE_METHODS: | ||
| # Moderators and node contributors can view actions | ||
| is_node_contributor = reviewable is not None and reviewable.node.has_permission(auth.user, osf_permissions.READ) |
There was a problem hiding this comment.
This is tight coupling with PreprintService (i.e. assuming node is present), but it's probably fine for now.
There was a problem hiding this comment.
Yeah, I agree. I kinda gave up on full decoupling at some point, pending the divorce. Are node permissions going to be generalized to apply to preprints too, or are we switching to something like guardian?
| @@ -0,0 +1,53 @@ | |||
| #!/usr/bin/env python | |||
| # -*- coding: utf-8 -*- | |||
| """Update auth groups for all review providers.""" | |||
There was a problem hiding this comment.
This should also be a management command.
| <ul class="dropdown-menu service-dropdown" role="menu"> | ||
| <li><a data-bind="click: trackClick.bind($data, 'Home')" href="${domain}">OSF<b>HOME</b></a></li> | ||
| <li><a data-bind="click: trackClick.bind($data, 'Preprints')" href="${domain}preprints/">OSF<b>PREPRINTS</b></a></li> | ||
| <li><a data-bind="click: trackClick.bind($data, 'Registries')" href="${domain}registries/">OSF<b>REGISTRIES</b></a></li> |
There was a problem hiding this comment.
Why was registries removed?
There was a problem hiding this comment.
My bad. Meant to remove "Reviews".
| preprint.node.is_preprint_orphan or | ||
| preprint.node.tags.filter(name='qatest').exists() or | ||
| preprint.node.is_deleted | ||
| not preprint._verified_publishable or |
There was a problem hiding this comment.
If _verified_publishable is public API, it shouldn't have the leading _.
There was a problem hiding this comment.
Also, _verified_publishable isn't exactly equivalent to the old code; is that expected? It looks like _verified_publishable doesn't check for preprint.node.is_preprint_orphan or preprint.node.is_public.
There was a problem hiding this comment.
It does check preprint.node.is_preprint, which seems like it covers those?
@property
def is_preprint(self):
# TODO: This is a temporary implementation.
if not self.preprint_file_id or not self.is_public:
return False
if self.preprint_file.node_id == self.id:
return self.has_published_preprint
else:
self._is_preprint_orphan = True
return FalseI think it might be more thorough than the old code? Downside is has_published_preprint makes a redundant query, but I figure it'll have to be reworked in the node/preprint divorce anyway, and it seems worth it to have all the "should this preprint be publicly visible?" logic in one place.
| def get_can_view_reviews(self, obj): | ||
| group_qs = GroupObjectPermission.objects.filter(group__user=obj, permission__codename='view_submissions') | ||
| user_qs = UserObjectPermission.objects.filter(user=obj, permission__codename='view_submissions') | ||
| return group_qs.exists() or user_qs.exists() |
There was a problem hiding this comment.
Minor: Can be slightly optimized with
return group_qs.exists() or obj.userobjectpermission_set.filter(permission__codename='view_submissions')|
|
||
| def get_reviewable_status_counts(self): | ||
| assert self.REVIEWABLE_RELATION_NAME, 'REVIEWABLE_RELATION_NAME must be set to compute status counts' | ||
| # TODO fix hackery once GUID query set is gone |
There was a problem hiding this comment.
GuidMixinQuerySet has changed dramatically this past week. Is this still an issue?
If the issue is the auto-included GUIDs, then you can remove the include using .include(None).
Something like
base_qs = getattr(self, self.REVIEWABLE_RELATION_NAME)
if isinstance(base_qs, GuidMixinQueryset):
base_qs = base_qs.include(None)
qs = base_qs.values('....')
There was a problem hiding this comment.
Yeah, that works nicely, thanks
d3ce398 to
0c93b75
Compare
|
@brianjgeiger @sloria Changes made! |
|
@aaxelb So moving the permissions to the permissions class didn't go well? |
|
@brianjgeiger Not really... It felt wrong to deserialize the response in |
09e50bc to
c4d34e4
Compare
|
@brianjgeiger I reconsidered -- permission logic in one place is definitely worth deserializing doubly. There might be a better way but this is better than what was. |
sloria
left a comment
There was a problem hiding this comment.
I don't see any remaining blockers.
| @@ -1,2 +1,6 @@ | |||
| class InvalidTransitionError(Exception): | |||
| pass | |||
| class InvalidTriggerError(Exception): | |||
|
|
||
| # overrides ListCreateAPIView | ||
| def get_queryset(self): | ||
| return Action.objects.none() |
There was a problem hiding this comment.
Rather than returning an empty queryset, would it make more sense to return 405 Method not allowed? @aaxelb @brianjgeiger
There was a problem hiding this comment.
I think I did that at first (used CreateAPIView instead of ListCreateAPIView), but it hid the browsable API, which felt wrong.
There was a problem hiding this comment.
We really aren't relying on the browseable API anymore, and once we are able to link to specific sections of the dev docs, all the BAPI docs will just be links. I'm good with this being CreateAPIView.
| embeds = [] | ||
| else: | ||
| embeds = self.request.query_params.getlist('embed') | ||
| embeds = self.request.query_params.getlist('embed') or self.request.query_params.getlist('embed[]') |
There was a problem hiding this comment.
Just curious: why was this necessary?
There was a problem hiding this comment.
If you're using jQuery.ajax (as Ember does) and assign an array to a query param, like say {embed: ['target', 'provider']}, it's serialized to ?embed[]=target&embed[]=provider.
The client-side workaround would be turning off deep serialization, but that breaks existing code that uses conveniences like { filter: { parents } } or { page: { size: 10 } }.
c4d34e4 to
7d19b57
Compare
| comment = models.TextField(blank=True) | ||
|
|
||
| is_deleted = models.BooleanField(default=False) | ||
| date_created = NonNaiveDateTimeField(auto_now_add=True) |
There was a problem hiding this comment.
Minor: Consider naming these created and modfied for compatibility with #7501
2ab7e40 to
cee05a7
Compare
This reverts commit 51d8b4d.
* State counts * Restore state counts on provider endpoint * plac8 flake8
* Show all preprints on My Projects page * Use new preprint filter for my projects page Capitalize status Don't show status initial
* [MOD-29] Reviews Initial Commit * [MOD-18] Add reviews to EXTERNAL_EMBER_APPS * [MOD-20][MOD-21][Feature] Add reviews mixins (CenterForOpenScience#21) * [MOD-20][Feature] Add PreprintService.state field * [MOD-21][Feature] Add ReviewProviderMixin * Responding to review * Use Enum * Isolate transitions machine * comments_public -> comments_private, for clarity * Log model and read-only endpoints (CenterForOpenScience#24) * [MOD-12][MOD-17][MOD-39]Setup ember-osf-reviews app (OSF-side) (CenterForOpenScience#22) * [MOD-12] Setup ember-osf-reviews app *Update docker setup *Add osf-reviews campaign *Add reviews route *Add osf-reviews logo *Update registration templates *Update osf navbar *Add mako landing page * fix * [MOD-22][MOD-24][MOD-25][MOD-48][Feature] Reviews API (CenterForOpenScience#26) * Add reviews fields to preprint/provider endpoints * Add permissions... * Cleaner permissions * Apply permissions to ReviewLog endpoints * Preprint permissions... * Use reviews permissions on all preprint endpoints * Filter preprint providers by reviews permission * Add provider setup endpoint * Allow node admins to see logs on their submissions Assuming the provider settings allow it * Hide log creators if comments are anonymous * Add script to generate fake ReviewLogs * Replace log state fields with action * Creating review logs to trigger state transitions * Allow resubmitting * Fix admin * Filter permissions correctly * Responding to review * Fix reviews migration * Responding to review * Fix stupid mistakes * Cleaner PermissionHelper * Fix provider log list view * Add preprint list permissions tests * Fix migrations after rebase * Fix a couple things... * No reviews workflow is None instead of 'none' * Create review logs at /v2/reviews/review_logs/ * Add read-only title and contributors to preprints * Hide preprints with private nodes from moderators * Add date_last_transitioned field to preprints * Expose live reload port for reviews * Better permission names * Tests for creating review logs, transitions * Expose date_last_transitioned in API (CenterForOpenScience#27) * [MOD-61] Set reviews settings by PATCHing provider (CenterForOpenScience#28) * Nullable provider settings, set all at once (CenterForOpenScience#29) * Add related counts for reviewables (CenterForOpenScience#30) * Add related counts for reviewables * Fixes * CR * [Fix] Hack around GuidQuerySet (CenterForOpenScience#31) * [Feature] More API tests and some cleanup (CenterForOpenScience#32) * More reviews tests * plac8 flake8 * Filter providers by multiple permissions with OR * Tests and cleanup... * More cleanup * Responding to review * Make travis happy * Post-rebase cleanup * Remove osf-reviews campaign * Make travis happier... * Add can_view_reviews to /users/me/ (#1) * [Feature][MOD-73] Rename log to action (#2) * ReviewLog => Action * Move Action model to osf.models * Fix up migrations * Fix failing tests * Fix up migrations again * Make /actions/ useful in the browsable API * Cleanup for consistency * Add `action` relationship to users serializer * Comment out reviews-specific dev docker stuff * Responding to review * Remove OSFReviews from navbar (#3) * Fix provider detail endpoint * Responding to review... * Keep permissions in permissions classes * Move scripts to management commands * Revert "Keep permissions in permissions classes" This reverts commit 51d8b4d. * Reenable sorting tests * Permissions in permission classes, mark 2 * Remove Reviews from the nav bar * Add email notifications * undo docker change * replace review_log with action * Fix style * Update test_notifications * Apply requested changes * Add tests * remove extra line * Change wording * Use preprint_word for providers * Apply requested changes * Remove print and fix space * Apply requested changes * Update test * Apply requested changes * Update test
…#8) * DOI minting * Remove white space * Remove check * Apply requested changes * Remove blank line * Update test * Apply requested changes * Apply requested changes
* Add migration for reviews subscriptions * Use iterator instead of paginator
5d11cc6 to
5244eaa
Compare
CenterForOpenScience#7769) * Add html version of emails Increase message max_length * Fix test
…CenterForOpenScience#7767) * Put all published preprints in `accepted` state * Fix failing tests * Fix migration conflict * Fix failing test * Fix failing test
…nce#7787) * Update email templates * * replace <br> with <p> * use provider domain * Apply requested changes
…terForOpenScience#7793) * Allow filtering preprints by the node's is_public Always filter out actions on preprints with private nodes from `/users/me/actions/` * Fix failing test * Clearer names * Fix typo
…nterForOpenScience#7778) * Update email notification for new submission * update test * Fix flake8 error * Apply requested change * apply another request * Update tests
|
Closed in favor of #7807 |
https://openscience.atlassian.net/browse/MOD-109
Purpose
All the backend changes required for OSF Reviews. Adds workflows for moderators to control which preprints are publicly available in their branded interface.
Changes
Actiontop-level objectsubmitted preprint X for review at time Y, changing its state frominitialtopending"/actions//actions/<id>//preprints/<guid>/actions//users/me/actions/ReviewableMixinfor objects which can be reviewedPreprintServicereviews_state: Current state of the preprintActionsReviewProviderMixinfor objects which own Reviewables and can choose a workflow for themPreprintProviderreviews_workflow: Provider's chosen workflow (see below)reviews_comments_private: Whether comments made by moderators are visible to the submitter/contributorsreviews_comments_anonymous: Whether the name/ID of the moderators who accept/reject/act on a preprint are visible to the submitter/contributors/preprint_providers/<id>/is once-writablereviews_workflowis non-null, the endpoint is read-onlypre-moderation: Preprints are publicly visible as soon as they're submitted, but can later be removed by moderatorspost-moderation: Preprints are visible only after approval by a moderatorinitial: Default state, created but not yet submittedpending: Submitted for and awaiting review, may be publicly visible depending on the workflowaccepted: Has explicit moderator approval, publicly visible under the providerrejected: Has explicit moderator disapproval, only visible to contributors and moderatorssubmitinitaltopending, or fromrejectedtopendingif resubmission is allowed by the workflowsubmitacceptpendingorrejectedtoacceptedacceptrejectpendingoracceptedtorejectedrejectedit_commentedit_commentadminandmoderatorgroups for each provider, assign group various object permissions on that providerSide Effects
Preprints workflow
If a provider has non-null
reviews_workflow, authors can no longer publish their preprints directly (by settingis_published). Instead, they must create asubmitaction at/actions/. The corresponding changes to ember-preprints are here: CenterForOpenScience/ember-osf-preprints#454If a provider has null
reviews_workflow, the existing behavior is unchanged. Workflows are entirely opt-in per provider, for now.Ember OSF
ember-osf changes: CenterForOpenScience/ember-osf#272